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I n recent years, the nation’s lowest-performing schools increasingly have 
become a focal point of scrutiny and concern. Policymakers have called for 
swift and dramatic action to improve the nation’s 5,000 lowest-performing 
schools, arguing that the magnitude of their dysfunction requires a robust 
response. Specific strategies for “turning around” chronically low-performing 
schools have become prominent, with the U.S. Department of Education 
enacting policies to promote four school improvement models that include 
“fundamental, comprehensive changes in leadership, staffing, and gover- 
nance” (State Fiscal Stabilization Fund Program: Final Rule, 2009, p. 58462). 
Spurred by federal grants and incentive programs, states and districts are 
attempting to catalyze rapid improvement in the lowest-performing schools 
through efforts that range from replacing principals, to firing the entire staff, 
to closing schools entirely. 

Despite the attention and activity surrounding these types of school 
improvement models, there is a lack of research on whether or how they 
work. To date, most evidence has been anecdotal, as policymakers have 
highlighted specific schools that have made significant test score gains as 
exemplars of school turnaround and researchers have focused on case studies 
of particular schools that have undergone one of these models. This has led 
to a tremendous amount of speculation over whether these isolated examples 
are, in fact, representative of turnaround efforts overall — in terms of the way 
they were implemented, the improvements they showed in student outcomes, 
and whether these schools actually served the same students before and 
after reform. 

THIS IS AN OVERVIEW OF FINDINGS, SUMMARIZED FROM A LARGER REPORT entitled Turning 
Around Low-Performing Schoois in Chicago: Fuil Report. For more information on the methodoiogy 
or findings of fhe study, piease visit our website at ccsr.uchicago.edu. 
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To begin to address this knowledge gap, the 
University of Chicago Consortium on Chicago School 
Research and AIR partnered to examine five different 
models initiated by the Chicago Public Schools (CPS) 
in 36 schools. CPS was an early adopter of dramatic 
intervention strategies in low-performing schools, and 
the reforms in this study were implemented between 
1997 and 2010. All of the schools were identified as 
chronically low performing and were reformed in ways 
consistent with the elements described in the school 
improvement models recommended by the federal 
government. The goals of the study were to make clear 
how school reform occurred in Chicago — showing the 
actual changes in the student population and teacher 
workforce at the schools — and to learn whether these 
efforts had a positive effect on student learning overall. 

SUMMARY OF STUDENT OUTCOMES 

• On average, Chicago elementary/ middle schools 
that underwent reform made significant improve- 
ments over time. Four years after intervention, the 
gap in test scores between reformed elementary/ 
middle schools and the system average decreased 
by almost half in reading and by almost two-thirds 
in mathematics. 

• On average, Chicago high schools that underwent 
reform efforts did not perform differently than 
similar schools in terms of absences in grades nine 
through 12 or in terms of the percent of students 
on-track to graduate by the end of ninth grade. 

Overview of Reform Models in Chicago 
Since 1997, CPS has initiated five distinct reforms that 
aim to dramatically improve low-performing schools 
in a short time. These initiatives are Reconstitution 
(seven high schools). School Closure and Restart (six 
elementary schools and two high schools), placement 
into the School Turnaround Specialist Program (STSP) 
model (four elementary schools), placement into the 
Academy for Urban School Leadership (AUSL) model 
(ten elementary schools and two high schools), and 
placement into the CPS Office of School Improvement 
(OSI) model (two elementary schools and three high 
schools).' 



All initiatives relied on changing the school leadership; 
this was the only lever of change under the STSP model. 
Administered by the University of Virginia s Partnership 
for Leaders in Education, the STSP program focuses 
on the leadership aspect of low-performing schools by 
training principals to be “turnaround specialists.” This 
reform is similar to the federal transformational model, 
where one of the requirements, among other elements, 
is the replacement of the principal in the school. 

Three other models relied on changing both the lead- 
ership and the school staff These are Reconstitution, 
AUSL and OSI models. Schools start the new academic 
year with dramatic changes to staffing, but the same 
students remain assigned to the schools. These reforms 
are similar to the federal turnaround model, which 
includes, among other actions, replacing the principal 
and at least 50 percent of the school’s staff, adopting 
a new governance structure, and implementing a new 
or revised instructional program. 

The last model. School Closure and Restart, was the 
most drastic intervention for several reasons: schools 
were closed for a year and students were moved into 
other schools; the schools subsequently reopened as 
charter schools; and student enrollment changed from 
assignment by neighborhood residence to an applica- 
tion and lottery system. In most cases, schools under 
the Closure and Restart model reopened with a few 
grades at a time and added a grade every year until the 
full grade structure was in place. This reform effort is 
similar to the federal restart model, in which schools are 
closed and reopened under the management of a charter 
school operator, a charter management organization, or 
an educational management organization. Table 1 shows 
some of the key elements of each of these strategies. 

The fourth federal model is school closure, where 
schools are closed and students are sent to other schools 
in the district. While some schools in the district were 
closed permanently and students were displaced to 
other schools, these schools are not studied in this 
report since the schools are permanently closed. A 
prior CCSR study examined the outcomes of students 
who attended schools that were closed; it showed that 
displaced students in Chicago tended to transfer from 
one low-performing school to another. Overall, closings 
had no effect on student learning for displaced students. 
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TABLE 1 

Five Reform Initiatives in CPS, 1997-2010 





Staff 

Replacement 


Leadership 

Replacement 


Governance 

Replacement 


Change in 
Attendance 
Ruies 


Sampie and 
Timing^ 


Reconstitution 


X 


X 






7HS 

(1997) 


Sctiooi Closure and Restart 


X 


X 


X 


X 


6ES;2HS 

(2002-2009) 


Sctiooi Turnaround Speciaiist Program 




X 






4ES 

(2006) 


Academy for Urban School Leadership 


X 


X 


X 




10ES;2HS 

(2006-2010) 


Office of Schooi improvement 


X 


X 






2 ES; 3 HS 
(2008-2010) 



^ ES = elementary school (schools serving any of the grades K through 8 but not serving students in the high school grades): HS = high school (sohools serving at least some of the 
grades 9-12). 



Main Findings 

Elementary/middle schools that went through reform made 
significant improvements in test scores compared with similar 
schools that did not; however, large improvements did not 
occur immediately in the first year. In the first year of reform, 
improvements in reading and math test scores were only 
marginally higher than those at comparison schools (see 
Figures 1 and 2) . But there was progress during the years 
after reform at almost all schools that underwent reform, 
in both reading and math, so that the gap in test scores 
between reformed elementary/middle schools and the 
system average decreased by nearly half in reading and 
by almost two-thirds in mathematics four years after 
intervention. (See Figures A1 and A2 in the appendix 
for school-by-school changes after reform.) These trends 
are net of changes in student population that the schools 
might have experienced. That is to say, the analysis 
adjusts for the fact that some schools did not serve the 
same students before and after the intervention. 



FIGURE 1 

Reading achievement in elementary/middle schools was 
significantly better after the second year of intervention; 
after four years the gap with the system average was 
reduced by almost half 



System Avg. 




■ Treated Schools ■ Comparison Schools 
-p<0.1 0, '*p<0.05, **p<0.01 and ***p<0.001 . Adjusted for changes In student backgrounds. 
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FIGURE 2 

Math achievement at elementary/middle schools was 
significantly better after the second year of intervention. 
After four years, the gap with the system average was cut 
by almost two-thirds 



System Avg. 




■ Treated Schools ■ Comparison Schools 
~p<0.10, *p<0.05, **p<0.01 and ***p<0.001 . Adjusted for changes in student backgrounds. 



FIGURE 3 

Ninth grade on-track rates in treated schools were not 
different from comparison schools 




■ Treated Schools ■ Comparison Schools 
''p<0.1 0, *p<0.05, **p<0.01 and ***p<0.001 . Adjusted for changes in student backgrounds. 



High schools that underwent reform did not show significant 
improvements in absences or ninth grade on-track-to-graduate 
rates over matched comparison schools, but recent high school 
efforts look more promising than earlier ones. 3 

On average, there were no significant improvements 
in ninth grade on-track rates and absence rates among 
the schools that went through intervention (see Figures 
3 and 4). While on-track rates have improved system- 
wide over the last several years, on-track rates did not 
improve more in schools that underwent reform com- 
pared to similar schools that did not undergo reform. 
There was a drop in absence rates in the first year after 
reform compared to matched schools, but the improve- 
ment was not sustained over time. 

Many of the high schools in this study went through 
the Reconstitution model in the late 1990’s, and this 
particular model did not lead to improvements in at- 
tendance or on-track rates. (See Figures A3 and A4 
in the appendix for school-by-school changes after 
reform.) CPS administration recognized the problems 
with earlier attempts at reconstitution, and many of the 
reconstituted schools were again targeted for reform in 
subsequent years. 

FIGURE 4 

High school absence rates were lower in the first year, on 
average, but not in subsequent years 




■ Treated Schools ■ Comparison Schools 
~p<0.1 0, *p<0.05, **p<0.01 and ***p<0.001 . Adjusted for changes in student backgrounds. 
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Methods for Studying Student Achievement Trends 



Reading and mathematics achievement trends for 
elementary schools (grades three to eight) as well 
as absences (grades nine to 12) and on-track- to- 
graduate (grade nine) trends for high schools were 
compared before and after the intervention took 
place. Two types of comparisons were used in what 
is called a difference-in-difference approach: 

1 . Comparing student performance in the schools 
before and after intervention; and 

2. Comparing trends in student performance in the 
reformed schools to student performance in a 
group of matched schools that did not experience 
the intervention. 

The matched group of schools had similar stu- 
dent performance as the reformed schools before 
intervention, and served similar types of students. 
Three different methods of choosing a matched 
comparison group were used to ensure that the 
conclusions would be the same regardless of which 
schools were used to make the comparisons. The 
analyses of student outcomes also took into account 
any changes in the background characteristics of 
students entering the schools over time, including 
changes in students’ race, gender, socio-economic 
indicators, prior achievement, grade level, age at 
grade, and disability status. 

Math and reading scores come from standardized 
tests that CPS students take each year in grades 
three through eight. Until spring 2005, the Iowa 
Test of Basic Skills (ITBS) was administered in 
CPS ; after that the test was replaced by the Illinois 
State Achievement Test (ISAT). To make the scores 



comparable across tests, we converted them into 
standard deviations from the system mean in each 
year and for each grade. Thus, they can be inter- 
preted as the degree to which students’ scores were 
different from average in the system in that year. 
For example, a score of -0.50 means that students 
are performing one-half of a standard deviation 
below the system mean for students in their grade. 
Standardizing within each year provides an auto- 
matic adjustment for any system-wide trends that 
should not be attributed to the intervention, or to 
differences in tests, or scoring of tests, that may have 
occurred across the years. 

Test scores cannot be used to study high schools 
because tests were given at different grade levels over 
the course of the years being studied. Therefore, 
absences and on-track rates are examined here. 
Student absence rates were recorded by CPS in a 
slightly different manner in the years prior to 2007- 
OS from how they were in the years after. In order 
to make measures comparable, we converted each 
student’s absence rate into standard deviations from 
the system mean for each fall semester. This makes 
the measure of absence rates consistent across all 
years included in this study; they can be interpreted 
as the degree to which students’ absence rates for the 
fall semester were different from the average in the 
system for that year. The on-track-to-graduate vari- 
able was not standardized because it was measured 
in the same way for all years in the study, taking a 
value of zero for students off-track and a value of one 
for students on track. A student who is on track has 
accumulated hve credits and has no more than one 
semester F by the end of ninth grade. 



Summary Report 



5 



More recent attempts at high school reform have 
paid more attention to school organization. As yet, 
there are only seven high schools that experienced re- 
form models other than Reconstitution, and several of 
them have only been in existence for one or two years. 
We are hesitant to make sweeping conclusions based 
on such a small number of schools with limited data. 
However, among those schools with at least one year 
of data, most showed some improvement in on-track 
rates above the comparison schools. (See Figures A3 
and A4 in the appendix for school-by-school changes 
after reform.) Most of the high schools reformed in 
recent years also showed a decline in absence rates in 
their first year, although not in subsequent years. 

Schools that underwent reform generally served the same 
students as before intervention, with the exception of one 
model of reform (see Figure 5). With the exception of 
schools in the Closure and Restart model, schools 
reenrolled between 55 and 89 percent of students who 
could reenroll in the year after intervention — rates that 
were similar to their year-to-year reenrollment rates 
prior to intervention. These patterns held true in the 
second and third years following intervention as well. 
In fact, more students reenrolled in subsequent years 
than in the first year of reform. The composition of 
students in intervention schools — in terms of race/ 
ethnicity, socioeconomic status, and special educa- 
tion status — was similar before and after intervention. 
These data contradict claims made by critics who argue 
that turnaround schools systematically push out low- 
performing and more disadvantaged students when 
schools undergo the transformation and turnaround 
models. Concerns about who the schools serve are valid, 
however, for the closure model. 

Schools under the Closure and Restart model experienced 
substantial changes to their student body composition, serving 
more economically advantaged students, students of 
higher prior achievement, and fewer special education 
students. After intervention, schools under the Closure 
and Restart model also served fewer students from the 
neighborhood around the school. 

The vast majority of teachers in schools under Closure and 
Restart, AUSL and OSI models were not rehired after reform. 
These schools rehired less than 10 percent of the teach- 
ers from the year before intervention. This is consistent 



with the theory of change behind the federal restart 
model and turnaround model, which requires that at 
least half of the staff change. In contrast, most schools 
in the Reconstitution model rehired about half of their 
teachers. Schools that were reconstituted had only a 
few months for planning and hiring new staff, and 
this may account for the larger percentage of teachers 
who were rehired. The teacher workforce after intervention 
across all models was more likely to be white, younger, and 
less experienced, and was more likely to have provisional cer- 
tification than the teachers who were at those schools before 
the intervention. 

How Should We View These Results? 

The results of this study suggest that turning around 
chronically low-performing schools is a process rather 
than an event. It does not occur immediately when staff 
or leadership or governance structures are replaced, but 
can occur when hard work and resources are sustained 
over time. 

Other studies have suggested that successful efforts 
to turn around low-performing schools usually do so 
by building the organizational strength of the school 
over time, using staff changes as just one of many 
mechanisms to improve school climate and instruction. 
A list of recommendations compiled in the lES Practice 
Guide on School Turnaround'^, based on case studies 
of schools that showed substantial improvement, starts 
with establishing strong leadership focused on improv- 
ing school climate and instruction, strengthening 
partnerships across school communities, monitoring 
instruction, addressing discipline, and building dis- 
tributed leadership among teachers in the school. The 
second recommendation is to maintain a consistent 
focus on improving instruction by having staff col- 
laborate around data to analyze school policies and 
learning conditions. The third recommendation is to 
pursue quick wins that target critical but immediately 
addressable problems, including student discipline and 
safety, conflict in the school community, and school 
beautification. The final recommendation is to build 
a committed staff, dedicated to school improvement 
through collaboration. None of the schools highlighted 
in the lES practice guide as successful examples of 
school improvement changed its entire staff, but all of 
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AUSL STSP Closure and Restart Reconstitution 



FIGURE 5 

Reenrollment rates in the first year of intervention were similar to the rates before intervention, with the exception of restarts 



O) 

o 



Percent of Students Re-enrolling Re-enrollment Prior 

0 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Comparable Time Period 

DuSable HS 
Robeson HS 
Harper HS 
Phillips HS 
Englewood HS 
King HS 
Orr HS 
Dodge ES 
Williams ES 
Howland ES 
Bunche ES 
Englewood ES 
Morse ES 
Frazier ES 
Collins HS 
Ames MS 
Earle ES 
Medlll ES 
Jackson ES 
Sherman ES 
Harvard ES 
Howe ES 
Orr HS 
Morton ES 
Dulles ES 
Johnson ES 
Bethune ES 
Curtis ES 
Deneen ES 
Bradwell ES 
Phillips HS 
Copernicus ES 
Fulton ES 
Harper HS 
Fenger HS 
Marshall HS 



I I I I I I I I I I L 



62.3% 



59.5% 



61.6% 



58.4% 



60.9% 



56.8% 




11.8% 




0.0% 


11.4% 




8.9% 





0 . 0 % 




58.7% 


67.0% 


62.9% 


50.9% 


65.9% 


59.7% 


61.7% 


57.1%* 


68.9%* 


38.7%* 


41.6%* 


28.6%* 


41.8%* 


60.2%* 


46.7%* 


76.9% 


69.8% 


68.6% 


74.7% 


73.3% 


65.8% 


66.0% 


67.7% 


52.1% 


64.0% 


61.6% 


76.0% 


72.4% 


60.3% 


52.9% 


65.2% 


65.5% 


54.2% 


70.2 %* 


71.2% 


64.1% 



*Re-enrollment rate at this school was significantly different after reform 
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them replaced teachers who did not share a commit- 
ment to change. 

This is consistent with research at CCSR examining 
100 elementary schools that made significant progress 
over a seven-year period — and 100 more that did not. 
The research found that schools strong on at least 
three of five essential elements — effective leaders, col- 
laborative teachers, strong family and community ties, 
ambitious instruction, and safe and orderly learning 
climate — were 10 times more likely to improve and 30 
times less likely to stagnate than those that were strong 
on just one or two.5 Perhaps it is not surprising, then, 
that the recent reform models, OSI and AUSL — both 
of which have explicit blueprints for reform focused 
on building the organizational strength of schools — 



achieved consistent improvement in all of the elemen- 
tary/middle schools they managed. 

Continued study will be needed to know whether 
these gains are sustained beyond the first four years, 
particularly if attention and resources from the district 
start to fade. These schools started out with extremely 
low levels of student performance and presented sig- 
nificant barriers to reform. A prior study at CCSR 
showed that over the past 20 years, CPS schools that 
started out with the lowest performance showed the 
least improvement.*^ From this perspective, this study 
provides promising evidence about efforts to improve 
chronically low performing schools — showing im- 
provements in schools that historically have been most 
impervious to reform. 



Endnotes 

1 . This office was previously known as the Office of School 
Turnaround. AUSL is a local school management organization 
charged with the training of teachers to affect whole-school 
transformation. They partnered with CPS to transform 
low-performing schools. 

2. De la Torre, M., and J. Gwynne (2009). When Schools Close: 
Effects on Displaced Students in Chicago Public Schools. 

Chicago: Consortium on Chicago School Research. 

3. Because the district did not consistently administer tests to the 
same grade levels over the period being studied, we were unable 
to examine changes in test performance at the high school level. 



4. Herman, R., Dawson, P., Dee, T, Greene, J., Maynard, R., 
Redding, S., and Darwin, M. (2008). Turning Around Chronically 
Low-Performing Schools: A practice guide (NCEE #2008-4020). 
Washington, DC: National Center for Education Evaluation 
and Regional Assistance, Institute of Education Sciences, U.S. 
Department of Education. Retrieved from http://ies.ed.gov/ncee/ 
wwc/ publications/ practiceguides. 

5. Bryk, A.S., P. Bender Sebring, E.M. Allensworth, S. Luppescu, and 
J.Q. Easton (2010). Organizing schools for improvement: Lessons 
from Chicago. Chicago: University of Chicago Press. 

6. Luppescu, Stuart. Elaine M. Allensworth, Paul Moore, Marisa de la 
Torre, James Murphy with Sanja Jagesic. 2011. Trends in Chicago’s 
Schoob Across Three Eras of Reform, http://ccsr.uchicago.edu/ 
content/publications.php?pub_id= 1 57 
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Appendix 



School-by-School Changes in 
Student Achievement 

T he figures presented in this appendix come from the same models that 
estimate the overall effects on test scores. These figures are adjusted 
for the same changes in students’ characteristics: changes in students’ race, 
gender, socio-economic indicators, prior achievement, grade level, age at 
grade, and disability status. In each figure, the bars on the left represent the 
difference in the first year after reform compared with similar schools that did 
not undergo intervention. Bars that are positive indicate more growth than 
comparison schools. Bars that are negative indicate less growth. The bars on 
the right in each figure represent the difference in the yearly growth rate in 
the years after the first year, up to the fourth year after reform. Schools that 
were reformed less than four years ago have fewer years on which to calculate 
their yearly growth after year one, and the bars represent the average of the 
years they existed. To find the cumulative effect, the yearly growth should 
be multiplied by the number of years since the first year of reform, plus the 
year one effects. 
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FIGURE A1 

Improvement in reading scores was variable in the first year, but consistently up in later years in almost all schools 

Reading growth at elementary/middle schools that underwent reform 
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Difference in Yearly Growth in Years 2-4 
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NOTE: Units are in standard deviations, adjusted for changes in student backgrounds. The comparison is to matched schoois. Positive numbers indicate more 
growth. The schooi-by-schooi figures are based on the popuiation of students at each schooi. Schooi names are the name before intervention. 
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FIGURE A2 

Improvement in math scores occurred at most elementary/middle schools that underwent reform 

Math growth at elementary/middle schools that underwent reform 

Difference in Yearly Growth in Years 2-4 



O) 

o 



- 0.05 



Difference in Year One 
0.00 0.05 0.10 0.15 



Dodge ES 
Williams ES 
Howland ES 
Bunche ES 
Morse ES 
Frazier ES 



Ames MS 
Earle ES 
Medill ES 
Jackson ES 



Sherman ES 
Harvard ES 
Howe ES 
Morton ES 
Dulles ES 
Johnson ES 
Bethune ES 
Curtis ES 
Deneen ES 
Bradwell ES 



Copernicus ES 
Fulton ES 




0.20 

j 



- 0.05 



C/5 
I — 
C/5 



C/5 

=) 

< 



O) 

o 



Dodge ES 
Williams ES 
Howland ES 
Bunche ES 
Morse ES 
Frazier ES 




Ames MS 
Earle ES 
Medill ES 
Jackson ES 



Sherman ES 
Harvard ES 
Howe ES 
Morton ES 
Dulles ES 
Johnson ES 
Bethune ES 
Curtis ES 
Deneen ES 
Bradwell ES 



Copernicus ES 
Fulton ES 



0.08 



0.06 



0.07 




0.09 



0.10 



0.09 



0.07 



0.08 



0.07 



One year of data 
One year of data 
One year of data 



No data 
No data 
No data 




NOTE: Units are in standard deviations, adjusted for changes in student backgrounds. The comparison is to matched schoois. Positive numbers indicate more 
growth. The school-by-schooi figures are based on the popuiation of students at each schooi. School names are the name before intervention. 
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AUSL and Restart Reconstitution 



Appendix 



FIGURE A3 

There was large variation across high schools in improvements in absence rates 
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FIGURE A4 

Recent reform efforts showed more improvements in on-track rates than eariier efforts 



On-Track Rates at High Schools that Underwent Reform 
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No data 
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NOTE: Units are in percentages, adjusted for changes in student backgrounds. The comparison is to matched schoois. Positive numbers indicate more growth. 
The schooi-by-schooi figures are based on the popuiation of students at each school. Schooi names are the name before intervention. 
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